DOCUMENT RESUME 



ED 046 373 



JC 710 024 



AUTHOR 


Brightman, Rich 


TITL^ 


Project CATCOP. 


INSTITUTION 


Coast Community 


PUB DAT^ 


Dec 7 o 


NOTE 


34d. 


FURS PRICE 


EDRS Price ME- $ 


DESCRIPTORS 


Classroom Techn 
Instruction, *T 
Media, ^Junior 


IDENTIFIERS 


Calf if or nia 



ard W . 

Final Deport. 

Coll. District, Costa Mesa, Calif. 



0.65 HC- $3.2° 
icrues, *Computer Assisted 
ndependent Study, *Tn str uctional 
Colleges, *Lav Enforcement 



A* F STR ACT 



This is the final report of project CALCOP, a joint 
project of the Coast Community Colleqe District, the Los Angeles 
Police Department, and the Los Anqeles Police Academy to develop, 
implement, and evaluate computer-assisted instruction (CAT) in the 
rules of evidence and seach-and-seizure procedures of police 
training. The CALCOP learning system is a combination of independent 
study and C? I exercises. The project compared one group of cadets 
usinq the CAICOP learning system and another qroup with similar 
characteristics using conventional classroom instruction. CALCOP 
provided simulated case problems for cadets that reinforced learning, 
broadened perspectives, and provided a mock field experience. The 
conclusion, based on examination results, shows that a learning 
system that removes the cadet from the rigid discipline of the 
conventional academy classroom is more effective as a teachinq 
technique than the traditional classroom. (CA) 




St 7/0 oa*A EDO 46373 



US OEPAATMENTOF HEALTH. EDUCATION 
* WELFARE 

OFFICE OF EOUCATfON 
THIS DOCUMENT HAS BEEN REPRODUCED 



e district 



3370 ADAMS AVENUE * COSTA MESA * CALIFORNIA 92626 



EXACTLY AS RECEIVED FROM THE PERSON OR 
ORGANIZATION ORIGINATING IT POINTS OF 
VIEW OR OPINIONS ST AT ED DO NOT NECES 
SARILY REPRESENT OFFICIAL OFFICE OF EDO 
CATION POS:TION OR POLIO 



f 1 

\Joast C M 



oast \Jommunity Uolleg 



Col 




r£Q 0 3 1971 



CLEARINGHOUSE FOR 
JUNIOR COLLEGE 
INFORMATION 



PROJECT CALCOP 
FINAL REPORT 
DECEMBER, 1970 



Richard W« Tirightnan 
Director, Research and Planning 
Coast Community College District 
1370 Adams Avenue 
Costa Mesa, California 92626 



0 ORANGE COAST COLLEGE 

£ COSTA MESA 

ERIC 



0 GOLDEN WEST COLLEGE 

HUNTINGTON BEACH 



1. 




EDO 46373 



PREFACE 



This report describes the activities of Project CALCOP. a Joint project 
of the Coa3t Comunity College District* the Los Angeles Police Department, 
and the Los Angeles Police Academy. The project was financed in part by a 
grant frurn the Law Enforcement Assistance Administration (Institute Grant 
NI-066) , and this paper serves as the final report of the project • 

A number of Individuals deserve recognition for their efforts in doing 
the work of the project: 



Hr. Deraid D. Hunt, Director of Law Enforcement Program for Golden West 
College, for designing and preparing the Study Syllabus and the computer 
simulated case problems and for scoring the final examinations. 

Sergeant M. R. Ingalls, of the Los Angeles Police Academy, for designing 
and testing the final examination and for reviewing the Syllabus and 
other training materials. 

Mr. Monty Ruth, of the Coast Community College District, for preparing 
and implementing computer programs used in the simulation exercises and 
in the statistical analysis. 

Sergeant Diane Karber, of the Los Angeles Police Department, for coordin- 
ating the otherwise diverse efforts of the Los Angeles Police Academy 
and the Coast Community College District. 

Miss Bonnie Borawski and Mrs, Ellen Gradick, of the Coa9t Community 
College District, for their efforts in assuring that the study 
materials and this report were properly produced. 

I list here others whose help represent important contributions to the 
success of this project: Lieutenant Delbert R. Wheaton, of the Los Angelas 
Police Department; Officer Ray Heslop, of the Los Angeles Police Department; 

Mr. George Martin, of the Los Angeles Police Department; Mr, Thomas Adams, 
Coordinator of the Police Science Program at Santa Ana College; and Officer 
Roger Sobie, of the Los Angeles Police Departcnent. 
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I. PROJECT CALCOP SUMMARY 

Coast Community College District and the Los Angeles Police Department 
have completed a joint project for the development, implementation and evalua- 
tion of computer assisted instruction techniques in a specific area of police 
training. 

Recent months have seen considerable excitement concerning computer 
assisted learning as a new instructional technique. By and large, computer 
assisted learning, or as it is often called, computer assisted instruction 
(CAI), is defined as a process in which a student Interacts more or less 
directly with a computer system in a learning situation* 

PURPOSE 

Project CALCOP served a two-fold purpose. First, the project sought to 
develop a computer assisted learning system for the purpose of training in 
the area of search and seizure and rules of evidence. Second, the project 
evaluated the effectiveness of the computer assisted learning system. In 
doing this, the project examined the hypothesis that the learning system 
designed by the project, consisting of independent study and CAI exercises# 
would be more effective than conventional classroom instruction. 
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PROCEDURES 



Procedures followed In Project CALCOP, are enumerated below; 

1- Objectives of training programs in search and seizure and 
rules of evidence were formulated.- 

2. An examination designed to teat the degree to which the 
objectives were met was developed. 

3. A syllabus of cognant material to be used for study purposes 
on an independent basis was prepared. 

A. Case problems simulated through the use of the computer 
terminal were prepared and implemented. 

5. Training was conducted U9ing the computer assisted learning 
system and the syllabus at Golden West College. Training 
also took place through conventional classroom instruction 
at the Los Angeles Police Academy. 

6. The examination was administered to police cadets at both 
the Los Angeles Police Academy and the Golden West Academy. 
Performance on this examination was compared between the two 
groups to determine if the computer assisted Instruction 
techniques were more or less effective than conventional 
classroom techniques. 

RESULTS 



Comparison of examination performance levels on the part of the Los 
Angeles Police Academy cadets and the cadets at Golden West College Police 
Academy showed that the Golden West College group performed significantly 
better on each of the three parts of the examination as well as for the 
examination as a whole. The difference in performance levels was found to be 
statistically significant in each case at the .01 level of confidence. 

CONCLUSIONS 

Learning systems such as that developed by Project CALCOP which remove 
the police cadet from the rigid discipline of the academy classroom show 
significant promise as more effective pedagogical techniques than current 
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II. PROCEDURES 



Project CALCOP engaged in a number of activities during its execution. 
These include establishment of behavioral objectives to be achieved by police 
cadets using the learning materials developed; establishing a steering 
committee for the project; establishing an executive committee for the 
project; preparing the simulation materials? and designing, testing and ex- 
ecuting evaluation methods. Each of these activities is discussed in the 
paragraphs to follow. 

ESTABLISH PROJECT STEERING COMMITTEE 

As outlined in the project proposal of April 10, 1969, Project CALCOP 
operated under the guidance of a steering committee composed of police 
officials, educational experts, and lay police advisors. Individuals serving 
on the Project CALCOP steering committee are listed below. 

Inspector George Beck, Assistant Commander, Office of Special 
Services, Los Angeles Police Department, Chairman. 

Dr. Norman E. Watson, Chancellor, Coast Community College District. 

Deputy Chief Robert Gaunt, Commander, Planning and Fiscal Bureau, Los 
Angeles Police Department. 

Inspector Vernon Hoy, Assistant Commander, Personnel and Training 
Bureau, Los Angeles Police Department. 
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Mr. Arthur Suchesk, Manager of Instructional Media and Systems, 

Southern California Regional Occupational Center. 

Mr. John S. Owens, Vice Chancellor, Vocational Education, Coast 
Community College District. 

Captain George Conroy, Commander, Records and Identificacion 
Division, Los Angeles Police Department. 

Mr. Deraid D. Hunt, Director of Law Enforcement Program, Golden 
West College Police Science Program. 

ESTABLISH PROJECT EXECUTIVE COMMITTEE 

The Project Executive Committee oversaw the work done by the project, 
determined goals and objectives, and reviewed the final results. The 
Executive Committee consists of police officers and educators as listed below. 

Lieutenant Delbert R. Wheaton, Los Angeles Police Department. 

Sergeant Diane Harber, Los Angeles Police Department. 

Sergeant M. R. Ingalls, L 09 Angeles Police Department. 

Mr. Deraid Hunt, Director of Law Enforcement Program, Golden West 
College. 

Mr. Richard W. Brightman, Director of Research and Planning, Coast 
Community College District. 

ESTABLISH GENERAL AND BEHAVIORAL OBJECTIVES 

The initial Project CALCOP proposal outlined broad objectives to be 
served by the Project. The first task of the Executive Committee, meeting 
during the summer of 1969, was to develop specific general and behavioral 
objectives of the program. These objectives are described in a later section 
of this report. 

PREPARATION OP STUDY SYLLABU S 

A syllabus was prepared outlining the factual or cognate material that 
Golden West police cadets should master before entering the field as operating 
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police officers. Preparation of this document involved the efforts of the 
Law Enforcement staff at Golden West College. The completed syllabus was 
thoroughly reviewed by the instructional staff at both Golden West College 
and the ios Angeles Police Academy. The review revealed several points in 
the syllabus that require updating and revision because of recent court 
decisions regarding police procedures in arrest, search and seuzire. A 
syllabus critique prepared by the Los Angeles Police Academy is available. 
Interested parties should address requests to: 

Richard W. Brightman 

Director of Research and Planning 

Coast Coiamunity College District 

1370 Adams Avenue 

Costa Mesa, California 92626 

In general, use of the syllabus by Golden West College police cadets 
pointed out the necessity for its continual review and updating. For this 
reason, the syllabus as shown in this report should not be viewed as a final 
document ready for distribution to law enforcement students, but rather as 
the first of a series of progressively updated documents outlining matters of 
arrest, search and seizure and rules of evidence. The syllabus appears in 
Appendix I of this report. 

PREPARATIO N OF SIMULATED CASE PROBLEMS 

Case problems simulated through the u9e of computer terminals were 
developed for twenty-six cases reported in the Law Enforcement Legal Infor- 
mation Bulletin published by the Los Angeles District Attorney's office. Use 
of these case problems involved a two-fold process* First, police cadets 
would aprise themselves of the basic facts of a particular case situation. 
Once satisfied that they were familiar with it and with the laws surrounding 
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the situation as presented in the syllabus, they would approach a computer 
terminal, identify themselves and the particular case they wanted to work on. 
The computer terminal would respond by asking them questions about the case, 
providing them additional information, and evaluating the results of their 
work. 

Appendix II includes all of the written descriptions of the twenty-six 
case problems as well as a list of all of the case problems identified by 
number and by the APL workspace name in which the cases could be found in the 
Coast Community College District computer system. Computer programming for 
the simulated portions of the case problems was accomplished through the use 
of APL programming language. Complete program documentation of each of the 
case problems is available from the Coast Community College District and 
interested parties should send requests to the address shown on the preceding 
page. 

Appendix III shows typical computer terminal output for the execution of 
cases 12 and 22. For the purpose of illustrating the manner in which 
incorrect responses were treated by ihe computer, the opeiator answered 
q estions incorrectly about as many times as he answered them correctly. 

PREPARATION OF EVALUATION MATERIALS 

In considering techniques of evaluation, the Executive Committee 
recognized the need to approximate, as much as possible, actual field 
situations that prospective peace officers are likely to encounter while on 
duty. Ideally, each cadet should investigate a mock field situation pre- 
pared by the educational institution and would be evaluated in terms of his 
performance in conducting his investigation. Clearly, this ideal evaluation 
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technique is impractical for most educational institutions, as it requires 
considerable amounts of time for each student being evaluated, A promising 
alternative, investigated by the Committee, involved depiction of one or more 
field situations through the use of photographic slides and/or video tape. 

Such presentation could be made to an entire class at once with the students 
answering specific questions concerning the situation as a means of taking 
the examination. Our investigations showed that with the resources avail- 
able to Golden West College, production of photographic slides or video 
tapes for use as described above was impractical. 

As a more feasible alternative, a written final examination was prepared 
using the same conceptual logic as might be used in a video tape presentation. 
A specific situation was described, questions were asked of the student 
about the situation and the student’s responses were evaluated to determine 
a test score. The examination prepared was tested thoroughly at the Los 
Angeles Police Academy before it was Implemented and administered to the 
control and experimental groups. This examination appears in Appendix IV. 

EVALUATION OF LEARNING MATERIALS 

The learning materials, consisting of the syllabus and the simulated case 
problems, were evaluated using established statistical and experimental 
techniques. These procedures are thoroughly described in Section IV of this 
report. 
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III. OBJECTIVES OP PROJECT CALCOP 



As reported in the Project CALCOP quarterly progress report of October 1, 
1969, and as later refined, the general and behavioral objectives of the 
project are enumerated below. 

GENERAL OBJECTIVES 



1* Develop study materials in search and seizure to be 
used for recruit training in criminal investigation; 

2. Develop computerized case problems which stem from 
(1) above and which reinforce learning, broaden per- 
spectives, and provide simulated field experiences for 
those completing the search and seizure section of 
recruit training; and to 

3. Evaluate the effectiveness of the learning materials 
developed in (1) and (2) above as compared with con- 
ventional classroom instruction in the same subject 
areas. 



These general objectives serve the broader purposes of: 

1. Preparing officers for field police work. 

2. Preparing officers to apply basic rales of evidence 
to field situations involving criminal investigation. 
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behavioral objectives 

After completing the segment of study prepared by Project CALCOP, 
police officers and police cadets 9 hould be able to perform the following 
tasks: 

1. Recognize Evidence and Identify Types of Evidence 

Demonstration of the ability to perform this task will involve 
studying a field situation and selecting and identifying 
pertinent evidence related to the situation. Within ten 
minutes , students will correctly identify 80 percent of the 
pertinent items of evidence found in an actual situation as 
examined through the use of written case descriptions and/or 
audio-visual presentations, 

2, Gather and Preserve Evidence 

a. Prepare Reports and Field Notes Demonstration of the 
ability to do this will involve studying field situations 
and identifying evidence to be included in specific 
report types. Within fifteen minutes students will 
examine a field situation and prepare reports required 
by the evidence on hand. The situation will be 
presented through the use of written case description 
and/or audio-visual presentation. 

b. Gather Testimony from Witnesses Demonstration of the 
ability to do this will involve identifying witnesses 
to a field situation who should be interviewed. 

Students will examine a field situation and within ten 
minutes must identify all witnesses who should be 
interviewed. The field situation will be presented 
through the use of written case descriptions and/or 
audio-visual presentation. 

c. Gather and Preserve Physical Evidence Demonstration 
of the ability to do this will involve identifying 
artifacts to be gathered from field situations as 
evidence and selecting means to collect and preserve 
them. Students will examine a field situation and 
list 85 percent of the items that should be gathered 
as evidence and will associate these with written 
descriptions of the means best used to gather and 
preserve them. This will be accomplished in twenty 
minutes. The field situation will be presented with 
written case descriptions and/or audio-visual 
presentation. 
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3 . Exercise Evidence-Gathering Techniques that Assure the 

Admissibility of the Evidence in Court 

Demonstration of the ability to perform this task will involve: 

a. Distinguishing evidence from non-evidence 
in field situations. 

b. Identifying evidence as found in field situations 
that will be inadmissible in court as opposed to 
that which will not be admissible. 



The student will examine a field situation and list items of 
evidence as differentiated from non-evidence and will further 
categorize items of evidence into those that will be excluded 
as opposed from those that would not be excluded in a court 
of law. Eighty-five percent of the items in the situation 
must be correctly categorized within twenty minutes. The field 
situation will be presented using written case descriptions 
and/or audio-visual presentation. 



MEE TING THE OBJECTIVES 

As originally articulated in the Project CALCOP proposal and in sub- 
sequent quarterly reports, the project's objectives pointed to considerably 
more elaborate learning systems than were feasible for development with the 
resources available to the District. For example rather than preparing 
elaborate tutorial interactive materials for computer-assisted study of 
cognant material in the area of search and seizure, the project found it 
more feasible to develop the study syllabus found in Appendix I. A syllabus 
was determined to be more flexible for student's use inasmuch as it could be 
used and studied virtually anywhere without requiring the student to use a 
computer terminal. 

The specific behavioral objectives found in Section III of this report 
were particularly difficult to evaluate in terms of the time available for 
evaluation. There is little question that the syllabus and the simulated 
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case problems as learning strategies contribute to the police cadet's 
ability to recognize, identify, gather, and preserve evidence in s manner 
that assures admissibility of the evidence in court. Designing evaluation 
devices to measure the degree to which these objectives are served by the 
learning strategies is quite a difficult matter* The total amount of 
classroom time typically spent in the area of search and seizure seldom 
exceeds ten hours* Testing exercises sufficient to measure the behavioral 
objectives outlined in Section III of this report must necessarily be very 
comprehensive and very detailed in nature, involve considerable photographic 
representation of case situations and probably would be best impelmented 
througih the use of a crime-site mock-up. Surrendering to the difficulties 
of preparing such evaluative instruments, we developed the examination 
appearing in Appendix IV as an approximation to the ideal expressed in the 
behavioral objectives* More about this important matter will be said in the 
conclusion of this report* 

Despite the difficulties in preparing an evaluative technique that meets 
the aspirations of the expressed behavioral objectives of the project, 
evidence presented in Section IV of this report leads us to believe that 
these instructional techniques are more effective in meeting the objectives of 
course work in search and seizure than in conventional classroom techniques. 
The examination that has been employed does in fact present the police cad^t 
with a case situation in which he must evaluate appropriate steps to take. 

His answers to the questions put to him by the examination are some in- 
dication of the degree to which he understands the appropriate procedures 
to use when actually in the field. 
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IV. EVALUATION OF THE LEARNING PROCEDURES 
STATISTICAL PROCEDURES 

Evaluation of the learni;^ procedures designed as part of Project CALCOP 
followed conventional statistical procedure. We were interested In the null 
hypothesis that there would be no significant difference In performance levels 
between cadets at the Golden West Police Academy (the experimental group) and 
cadets at the Los Angeles Police Academy (the control group) as measured by 
the examination enactments shown In Appendix IV. Finding a statistically 
significant difference would give us cause to reject the null hypothesis , 
concluding that the CAI learning procedures were either more or less effective 
than the conventional procedures , depending upon the sign of the difference. 

Comparison of performance scores between the control and experimental 
groups with respect to the CALCOP examination enactments as well as on the 
California Short Form Test of Mental Maturity and the Wonderlic Personnel 

Test made use of the t test for significant differences In mean scores* and 

•m* 

1 2 
the WllcoXoa matched pairs signed-rank test. 



*Fergeson, G. A* » Statistical Analysis In Psychology and Education . 
(New York* McGraw-Hill), pp. 167-174. 

?Selgal, S. , Non-Paraoetrlc Statistics for the Behavioral Sciences , 
(New York: McGraw-Hill) « pp. 75-83. "* ~~ ~ ~ ~~ 
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In comparing mean performance scores we used one of two calculation 
procedures to arrive at t, depending upon the homoscedastlclty of the teat 
score distributions of the two groups being compared. For those cases In 
which the variances were homogeneious , we used the formula 



t 



X 



1 



X 

2 




2 2 

Ete -X) + *<*-!> • 

11 2 2 



N + N - 2 
1 2 




Where X Is the Individual score for members of Group 1, X the 
1 2 

individual scores of the members of Group 2, X^ and the representative 

mean scores of Groups 1 and 2, the total number of students in Group 1, 

and N the total number of students in Group 2.^ 

2 

In those cases in which the variances of the two distributions the 
means of which were to be compared were not homogeneious , we used the 
formula 



t 



* 



% * 
*i 1 



2 



+ 



t 

2 







V 

2 



3 

Freund, J, B., Modem Elementary Statistics , Third Edition, (Englewood 
Cliffs, New Jersey: Prentice Hall), p. 256. 
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Where t^is the critical value of required for significance at the 
•05 level of confidence with Nj-1 degrees of freedom and the critical 
value of required for significance at the .05 level of confidence with 
N^l degrees of freedom and where 

X - X 
l 2 




with i * 1,2 and where 




N 



£(X -X > 






£(X -X ) 
+ 2 2 



N (N -1) 
2 2 



* \ 



S- + s_ 
x x 
1 2 



A 

where the variables are as described above. 

We tested the score distributions on each of the tests administered for 

horoogeniety of variance by considering the ratio of the two variances as 

2 

calculated by E(X - X ) 

' N - 1 

R » 1 

r(x - X > 2 

2 

M - 1 
2 



O 
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*Fergeso>n» o£i clt. , pp. 171-172. 
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and consulting a table of the F distribution for R to determine whether or 

not the difference between the variances is significant.^ In those cases in 

* 

which the variances were not homogenelous , t was calculated, otherwise we 

found t. Hereafter in this report, tests of significant mean differences will 

* . 

be reported as significant in terms of t or t depending upon the 

homoscedastlcity of the two distributions yielding the means. 

Use of the t (or t ) test for significance of mean differences requires, 

in addition to homogeneity of variance, that the distributions be normally 

distributed. Usually, with N * 30, normality may be assumed.^ However, as 

our populations never exceeded 28 in number and on one occasion was only 

eight, we performed the Wile oxen ranked-pairs test to verify that the 

significant differences ve found with the t and the t* tests also appeared 

significant under the weaker yet distribution- .^ree non-pa rame trie test. In 

* 

every ca9e, the Wilcoxon test yielded results that agreed with our t and t 
calculations. 

Evaluation procedures and the results of statistical calculations are 
described in the paragraphs that follow. 

EXAMINATION DEVELOPMENT 

Inasmuch as the purpose of the evaluative phase of Project CALCOP was 
to measure the relative effectiveness cf the computer assisted instruction 
techniques used with conventional classroom presentation techniques, a first 
important task of the project was to develop the final examination as appears 
in Appendix IV. 



5 Ibid., pp. 181-183. 
^Fr.und, op. clt. t p. 255. 
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The examination was tested at the Los Angeles Police Academy. Groups 
of cadets at the Academy would take the examination. After scoring, the 
cadets and the instructor would critique the examination in terms oi clarity 
and legal accuracy. After caking appropriate modifications, the instructor 
would administer the examination to a fresh group of cadets and repeat the 
evaluation. In this manner, cadet reactions to and performance on the 
examination was carefully considered in subsequent revisions of the final 
examination. Revisions were retested as described above until the final draft 

i 

of the examination as appearing in Appendix IV was completed. 

The examination consists oi four case enactments each of which provide 
the cadet with certain information regarding a particular case situation. In 
every case, the case situation presented by the examination was similar to a 
real life situation with names of persons and of places changed to prevent 
students, to every extent possible, from recognizing the situation as one that 
he may have studied earlier. 

CONTROL AND EXPERIMENTAL GROUP SELECTION 

The experimental group for this study consisted initially of twenty-seven 
polico cadets enrolled in the Golden West College Police Academy during the 
Fall semester, 1970*71. This group undertook to study matters of search and 
seizure through independent, use of the syllabus and through the use of the 
computer assisted instruction simulation exercises described earlier in this 
report . 

The control group for the experiment consisted of police cadets at the 
Los Angeles Police Academy who undertook to study matters of search and 
seizure through conventional classroom Instruction as conducted at that 
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Academy. Sixty police cadets out of a class of seventy-one at the Los Angeles 
Academy took the final examination enactments. 

Members of both the control and the experimental groups took the 
California Short Form Test of Mental Maturity and the Wonderlic Personnel 
Test. Using the IQ scores achieved on the California Short Form Test of 
Mental Maturity for each of the twenty-seven members of the experimental group 
as a basis i twenty-seven members of the Los Angeles Police Academy gvr-up were 
selected so as to give twenty-seven matching pairs of cadets, one group each 
from the Golden West College Police Academy and from the Los Angeles Police 
Academy. Table I shows the initial populations of both the control group 
(Los Angeles Police Academy group) and the experimental group, (the Golden 
West College group) and the degree to which IQ scores differed as between 
two members of any one matched pair. The differences between the mean IQ 
scores of the twenty-seven members of the control group as compared to the 
twenty-seven members of the experimental gicv.p were evaluated through the use 
of the t distribution. This yielded a t score of 0.218 indicating no 
significant differences between the mean IQ scores between the control and 
the experimental groups. * 

After completing the training program and gathering performance data, 
there remained twenty-three matched Golden West - Los Angeles Academy pairs 
for whom complete data were available. These matched pairs and their 
respective California Short Form Test of Mental Maturity scores (IQ scores) 
appear in Table II. 

Differences in IQ scores as shown in Table II between the Los Angeles 
Police Academy control group, Group 1, and the Golden West College experi- 
mental group, Group 2, were compared using two techniques. As described 
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earlier, the t test was performed to assess the differences between mean 
IQ’s for the groups* This yielded a t score of -0.04. This score is not 
significantly different at the .05 level of confidence. We also pei formed the 
Wilcoxon matched pairs signed-rank test. This procedure yielded a T score 
of 43 with an N of 14 which demonstrated no significant differences between 
the matched pairs at the .05 level of confidence. 

Table 111 shows the relative Wonderlic Personnel Test scores for both 
the Golden West College experimental group and the Los Angeles Police Academy 
group. We performed the same tests on the Wonderiic score differences as we 
performed for the California Short Form Test of Mental Maturity scores. 

The t test for significant differences between mean Wonderlic scores yielded 
a t of 0.8 which showed that there was no significant difference between the 
mean Wonderlic scores between the control and experimental groups. The 
Wilcoxon matched pairs test yielded a T of 83 for an N of 19, again showing 
no significant difference at the .05 level of confidence. 

We were also interested in the degree to which the Los Angeles Police 
Academy control group, consisting of twenty-three selected metnbers, repre- 
sented the total seventy-one members of the Los Angeles Police Academy from 
whom the control group was drawn. The t test for differences in mean IQ 
scores yielded a t of -1.4 which was not significant (.05 level). Similarly, 
the t test was used to measure differences in mean scores on the Wonderlic 
examination between the twenty-three members of the Los Angeles control 
group and the total seventy-one member Los Angeles Academy group that took 
the test. In this case, the t score was -1.7, again not significant at the 
.05 level of confidence. 
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We also compared the Golden West College experimental group with the 
total seventy-one member Los Angeles group. The t test in this c.^se yielded 
a t score of -1.43 which was not significant at the .05 level for mean IQ 
scores. Similarly, the t score for the differences in mean Wonderlic scores 
was -0.83, again not significant at the .05 level of confidence, 

As a result of these calculations and comparisons we can make the 
following observations; 

1. There is no significant difference in mean IQ scores as 
measured by the California Short Fora Test of Mental 
Maturity between the twenty- three member experimental 
group at Golden West College and the twenty-three member 
control group at the Los Angeles Police Academy. 

2. There is no significant difference in mean Wonderlic 
Personnel Test scores between the control group and the 
experimental group. 

3. The control group of Los Angeles Police Academy cadets is 
a representative sample in terms of IQ and Wonderlic 
scores of the total seventy-one member group of Los Angeles 
Police Academy cadets. 

4. There is no significant difference in either mean IQ scores 
or in mean Wonderlic scores between the Golden West College 
experimental group and the total group of Los Angeles Police 
Academy cadeta. 

Accordingly, any differences to be found between performance levels on 
the examination enactments as between Group 1 and Group 2 cannot be attributed 
to differences in intellectual ability as measured by the California Short 
Fora Tast of Mental Maturity and the Wonderlic Personnel Test. Differences 
in performance levels on the final examination must be accounted for by other 
factors than differences in measured ability. 

TRAINING 



Police cadets at the Los Angeles Police Academy studied materials 
relating to eroper procedures in search and seizure matters under conventional 
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classroom Instruction. This instruction consisted of lectures and classroom 
discussions. As described earlier in this report, police cadets at the 
Golden West College Police Academy studied the same materials making use of 
the study syllabus and the computer assisted simulated case problems. This 
group received no classroom instruction. 

EXAMINING 

After completing the training program in search and seizure, cadets at 
both the Police Academy in Los Angeles and the Academy at Golden West College 
completed a written examination consisting or four case problems or enactments 
in which the student was asked specific questions about procedures and matters 
of fact relating to the situation described. The examination appears in 
Appendix IV. 

All of the examinations were scored by Deraid Runt, the Coordinator of 
the Law Enforcement program at Golden West College. Scoring was done by 
one individual to minimize to every extent possible differences in scoring 
procedures chat might arise should more than one person score the tests. Of 
the four enactments included in the final examination, only three were scored 
for the Los Angeles Police Academy group. This is so because the fourth 
enactment was returned to the students ar«d was therefore unavailable for 
scoring at the same time that the other three enactments were available. For 
this reason, only the first three enactments of the final examination have 
been used in this study to measure differences in performance levels between 
the Los Angeles Police Academy group and the police Academy at Golden Weat 
College. 
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RESULTS 

Tables IV, V, and VI shew the relative examination scores for enact- 
ments 1* 2, and 3, respectively. Maximum score possible for enactment 1 
was 11. Maximum score possible for enactment 2 was 9, and for enactment 
3, a maximum score of 10 was possible. Table VII shows the total scores on 
all three enactments for each of the control - experimental matched pairs. 

Both the t test and the Vilcoxon matched pairs signed-rank tests were 
applied to the performance scores on the examination enactment. Table VIII 
lists the results of these calculations. In every case, cadets at the Golden 
West College Police Academy performed better on the final examination than did 
cadets at the Los Angeles Police Academy and in every case the difference In 
performance levels was statistically significant at the .05 level of con- ‘ 
fidence. For enactment 1 Golden West College cadets averaged 2.17 points 
higher In performance scores than. did their counterparts at Los Angeles. For 
enactment 2 the difference In mean performance level was 1.52 points higher. 
Similarly, for enactment 3 Golden Went College cadets averaged 1.96 points 
higher than did the Los Angeles Police Academy cadets. For the three 
enactments taken together the Golden West College group averaged 5.65 points 
higher in performance scores than did the group at the Los Angeles Police 
Academy. 
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V. CONCLUSIONS 



The most obvious conclusion to be drawn from the procedures outlined 
above says that the learning procedures followed at Golden West College In 
the area of search and seizure were more effective than were the procedures 
followed at the Los Angeles Police Academy, at least as measured by the final 
examination enactments appearing In Appendix IV of this report « Testing and 
selection of the experimental and control group minimized differences In 
performance level that might arise as a result of differences in abilities 
between the two groups. Selection procedures exercised by the Los Angeles 
Police Academy and the several police agencies employing the Golden West 
College cadets probably minimized differences in educational level, reading 
skills and writing skills that would not also appear as differences in IQ 
and Wonderlic scores. There remains then the difference In training procedures 
between the two groups as a factor which would account for the differences 
In performance levels. 

The control group at the Ix>s Angeles Police Academy undertook training 
in the area of search and seizure with conventional classroom Instruction 
under rigid circumstances in which the learning situation was rather well 
structured. Instructors at the Los Angeles Academy lectured to the cadets, 
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described to them specific case situations, and elicited responses from 
members of the class as to what they would do or what should be done in a 
particular case situation. Cadets at the Golden West Police Academy program 
had no such classroom instruction and limited their efforts to studying the 
syllabus and ansvering questions put to them about specific case situations by 
a computer terminal. We assert, and our conclusions here are based upon the 
statistics reported above, that this basic difference in instructional 
approach accounts for the differences we find in performance levels between 
the Los Angeles Police Academy control group and the Golden West College 
Academy experimental group. 

As we analyzed our data, however, we became interested in other 
phenomena that might partially account, for some of the observed performance 
differences. Experimental bias, for example, is a common place falling in 
most experimental studies of this kind and there is some likelihood that it 
may have played a part in increasing the performance level of the experimental 
group. The experimental group and the control group were widely separated 
geographically and enjoyed no inter-group communication whatsoever. 
Nevertheless the group at Golden West College did know that their performance 
levels on an examination covering the areas of search and seizure would be 
compared with scores on the same examination earned by Los Angeles Police 
Academy cadets. This knowledge may have motivated the group to apply 
themselves morn assiduously to their studies and, to the extent that they did, 
the experiment was biased. However, we should point out that most classroom 
teachers turn to a number of devices and strategies to motivate students to 
study harder and whether or not the devices and strategic* employed by the 
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Los Angeles Police Academy instructors in thi9 area were more or lees effective 
as motivators than the knowledge on the part of Golden West College cadets 
that their performance was to be compared with another group* is a matter of 
conjecture. 

A second phenomenon which might play an even more important part in 
explaining differences in performance levels between the two groups had to do 
with the experimental group at Golden West College learning how to take the 
final examination. The case problem approach as employed through the computer 
assisted simulations presented materials and questions about the facts of 
cases in almost exactly the same manner as is found in the examination itself. 
Thus students studying the syllabus and then answering questions about specific 
case situations as posed by the computer terminal were In effect taking an 
examination not at all unlike the one they would take as a final measure of 
t'^eir achievement. In this way, they were learning how to take this type of 
examination. Cadets at the Los Angeles Police Academy, on the other hand, 
had no similar training experience. Their exposure to the presentation of 
case situation facts and then answering questions about the situation was 
probably a new one for them. To examine the degree to which this might be 
true, we compare the control group performance on a multiple choice examina- 
tion covering the area of search and seizure and rules of evidence with the 
performance of a preceding class on the same examination. 

At the completion of the Police Academy at Golden West College, all 
cadets took a multiple-choice final examination covering all phases of the 
Academy program. The experimental group in this study took this examination 
as did the Academy class that immediately preceded them. The examination 
consisted of a number of separate parts, three of which contained no test items 
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dealing with matters of search and seizure and rules of evidence. The 
remaining parts contained, among other things , twenty- five questions 
concerning search and seizure and rules of evidence. Being interested in the 
degree to which cadets in our experimental group at Golden West College did 
better or worse than did their predecessor class, we examined their relative 
performance on the multiple choice final examination for the complete academy. 
The results of our analysis appear in Table IX. This table presents the mean 
percentage scores earned on each of the three portions of the test that 
included no test items dealing with search and seizure, and rules of evidence 
as well as for the three sections taken together, sud those twenty-five test 
items that deal exclusively with search and seizure and rules of evidence. 

Our comparison of mean scores followed the procedures discussed earlier and 
the resulting t (or t*) scores also appear on the table. 

Of the five mean differences in exam scores shown in Table IX, only the 
mean differences on the twenty-five questions dealing with search and selzuve 
and rules of evidence is statistically significant (.01 level). Our control 
group, then, did significantly better than their predecessors on the search 
and seizure and rules of evidence portion of their final exam but performed 
only equally as well on those protions of the final examination that dealt 
with other matters. 

Reconsider the argument that the experimental group performed better on 
the examination appearing in Appendix IV as a result of having learned how 
to take this type of examination more effectively than the control group. 

This may be true. However, they also learned, apparently, how to take 
multiple choice examinations better than their predecessors, but only with 
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respect to questions dealing with search and seizure and rules of evidence* 

The data appearing in Table IX lead us to discount heavily the argument that 
periurmance differences we found between the control and the experimental 
groups can be largely explained away as the result of having learned how to 
take a particular type of examination. 

A third phenomenon that could explain performance differences between the 
control and experimental groups has to qo with the degree of experience as 
operating police officers that cadets may have had prior to entering the 
police academy. Several cadets at Golden West College had previous experience 
as police officers before enrolling. Only one of the cadets at the Los 
Angeles Police Academy had any experience before entering his training 
program. In an effort to isolate the effect which previous experience may 
have had upon differences in mean performance levels between the two groups, 
we eliminated all those matched pairs in which the Golden West College member 
had had more than a few days ^ior experience. The remaining matched pairs, 
their respective IQ and Wonderllc scores, as well as their performance scores 
on each of the three examinations enactments and for the total examination 
appear in Table X. Both the T test and the Wilcoxon matched pairs ranking 
test for this non-experienced sub-group showed that there were no significant 
differences at the .05 level of confidence between the Lob Angeles and the 
Golden West groups with respect to either the IQ scores or the Wonderllc 
scores. As with the large group analysis. Golden West College cadets performed 
consistently better on all three enactments and for the total examination than 
did the Los Angeles cadets. In every case the increased performance was 
statistically significant at the .01 level of confidence. On enactment 1 
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Golden West College cadets earned an average of 2 points higher than did the 
Los Angeles group. This mean difference was 1.7 points higher for the second 
enactment and 1.9 points higher for the third. With respect to the examination 
taken as a whole. Golden West College cadets performed better than did the 
Los Angeles cadets by a mean difference of 6.2 points. 

This examination of the non-experienced cadet pairs leads us to conclude 
that the experience enjoyed on the part of some of the Golden West college 
police cadets played no significant role in accounting for the overall 
increased performance levels of the entire twenty-thr«e man experimental 
group* 

An even more important factor that might well explain the performance 
differences we found may be that of removing the police cadet from the 
classroom. Typically, classroom learning situations in police academies is 
much more rigorous and much more structured than typical classroom situations 
found in other college areas. Discipline is more rigidly enforced and 
students may feel less free to investigate areas of interest to them than do 
students in such areas as say philosophy, literature, or even mathematics 
and physics, In this respect, police academy classrooms resemble military 
basic training caops, As a result, police academy programs nay be criticized 
as being non-conduaive to learning. Developing learning situations for specific 
skills and specific areas of conceptual knowledge in law enforcement and re- 
moving students from a rigidly disciplined classroom environment while they 
study these subjects may well prove to be more effective than current methods. 

Although we are not prepared on the basis of Project CALCOP to conclude 
that the computer assisted learning portion of the learning system devised 
is more effective then classroom Instruction, we do think that the total 
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learning system including independent study of the syllabus as well as computer 
assisted case problems, presents a more effective learning environment in 
the area of search and seizure than does conventional classroom instruction. 
This is not to say, of course, that conventional classroom instruction has 
been other than excellent in quality. In fact we cannot say, as a result of 
this study, that it has been good, bad, or indifferent. Rather, we have 
found evidence that Instructional effectiveness in search and seizure can 
be further improved through the use of learning systems similar to that 
developed by Project CALCGP. 
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VI. RECOMMENDATIONS 

We have never seen a research report that does not close with 
recommendations for additional research. This one will not conclude 
differently. It Is clear to us from the work we have done so far that 
independent study and computer assisted learning techniques can play a most 
important role in the training of police officers. What is needed most at 
the present time are better examination procedures that more adequately 
assess the ability of police officers to perform in the field. The written 
examination used as part of Project CALCOP may not serve adequately at all 
as compared to a more realistic evaluation procedure in which police officers 
investigate a mock-up crime situation. The first step, then, in continuing 
the type of study started with Project CALCOP is to engineer such evaluation 
devices and validate them as appropriate measures of operating skills on the 
part of active police officers. 

Other experimentation with computer assisted learning as well as that 
imdertaken with Project CALCOP has led us to believe that the typewriter 
terminal la an inadequate device for computer assisted learning. It would be 
much better, we think, to present written, photographic, or other graphical 
information to etudente in the form of visual display. This cannot be done at 
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the present time through the use of typewriter terminals such as those 
employed in Project CALCOP. Under investigation at this time by the Coast 
Community College District is the use of random access microfiche display 
units under the control of a computer. Combining the materials we have 
already prepared for Project CALCOP with microfiche display techniques , we 
think that we could substantially improve the learning system devised. Rather 
than read a written description of a case situation, students would instead 
study photographic images portraying the particular situation. In such a 
system the student would still enjoy the individualized attention that he 
currently receives from the computer terminal, however, he does not have to 
wait for the typewriter terminal to finish typing out 8 message before he 
can respond to it# Written messages as well as photographic information can 
be displayed on an Illuminated screen within a few seconds access time while 
the student continues to enter his answers into a computer typewriter terminal. 
Experimental work with this system is just beginning and we think that Project 
CALCOP has played a significant role in pointing us in this direction. 

Officials at the Golden West College Police Academy are Interested in 
pursuing the learning strategies employed in Project CALCOP in other areas of 
police training. This too is an lng>ortant area for continued study and 
research* An earlier study completed by the Coast Community College District 
found CAI to be equally effective as classroom instruction, but no better. 

We harbor strong suspicions that learning systems that remove the police 
cadet from the disciplinary atmosphere of the Academy classroom may alone be 



Computer Assisted Learnin g to Teach Computer Operations , Unpublished 
research report, November, 1970* 
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more effective than current techniques. We need to answer two questions in 
this regard. First, to what extent can the performance differences found 
by Project CALCOP be explained by the CA1 system as opposed to simply 
removing the stadent from the classroom for self-study? Second, is self- 
study in general (whether or not computer-assisted) a more effective instruc- 
tional stragety for police training than current classroom techniques? 
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